Computer systems, and particularly networked computer systems, have become complex. An aspect of this complexity is that, at times, computers systems may behave in unexpected ways. This generally causes consternation among computer system users and may prompt calls to a computer system support organization. One challenge for the support organization is determining whether the behavior at issue is unexpected in general, and possibly undesirable, or merely unexpected to a particular user.
The challenge is greater than might be anticipated at first glance. Support organizations typically have multiple layers with increasingly specialized experts. However, even high expertise with a particular computer system may be degraded by the constant release of new computer systems, sub-systems, services, and components, new versions of existing computer systems, sub-systems, services, and components, new interfaces to existing computer systems, sub-systems, services, and components, and the like. In such an environment, documentation may quickly become outdated and unreliable. The line between expected and unexpected behavior may become ambiguous, forcing the support organization to fall back on, at best, educated guesswork.
Rising above guesswork means being able to reproduce and analyze questionable behavior, but again this is easier said that done. One problem is translation between user-perceived categories of system functionality and categories of system functionality as implemented. Test cases designed to test categories of system functionality as implemented may be inappropriate and/or inadequate to test categories of system functionality as perceived by system users. An example of this is the now common case that multiple and layered interfaces, such as graphical user interfaces (GUI), command line interfaces (CLI), and application programming interfaces (API), exist to the same sub-system, service or component. A computer system user perspective of a set of interfaces may include multiple and/or different test cases relative to the perspective of a computer system implementer, and vice versa.